The traditional process of selling cosmetics products1 involves customers visiting makeup retail shops, requesting samples of cosmetics, which they, then, apply to assess the feel and look of the cosmetics on their face, hands or other parts of their bodies. This process is cumbersome for cosmetics retailers and customers alike. Cosmetics retailers must go to the expense of providing product samples and dedicated staff to apply them to customers. As the application and removal of cosmetics is time consuming, customers are limited in the number of samples they can physically try on in order to assess their suitability. It would be desirable, therefore, for there to be a system whereby customers might simulate the appearance of cosmetics on their bodies using a personal computation device fitted with a camera and screen. 1 No distinction is made among the terms “makeup,” “cosmetics,” “makeup cosmetics,” and “cosmetic products,” all of which are used interchangeably herein.
While, for heuristic convenience, the present application may refer to makeup and its application to the face, it is to be understood that the teachings herein apply in equal measure to any cosmetic product applied to any body part. Similarly, references to ‘makeup’ subsume other cosmetic product such as nail varnish, skin tanning lotion, permanent makeup and tattoos.
It would be particularly desirable for there to be a process for retailing cosmetics digitally, using methods described in copending U.S. Provisional Patent Application Ser. No. 61/874,506, filed Sep. 6, 2013, which is incorporated herein by reference. Similarly, it would be desirable to employ the methods of the Mojo Masks App, described on the Webpage, www.businesswire.com/news/home/20121022006659/en/Image-Metrics-Unwraps-Mojo-Masks-App (Oct. 22, 2012), appended hereto as Appendix A and incorporated herein by reference.
Methods for digital simulation of makeup can be broadly categorized into image-based and video-based techniques. Image-based methods provide single images of a person on which makeup is to be applied and work in an offline mode. Image-based makeup systems include those described in U.S. Pat. No. 6,937,755 “Make-up and fashion accessory display and marketing system and method,” which details a method for compositing different makeup products and applying it to an image. Another image-based makeup system is described in U.S. Pat. No. 8,265,351 “Method, system and computer program product for automatic and semi-automatic modification of digital images of faces” where the application of makeup involves combining the original image and the makeup image. The foregoing methods fail to account for the transparency properties of makeup. Both methods only take into account the overall illumination variation in the original image, without making corrections for local illumination variation.
It would thus be desirable to provide real-time video-based makeup simulation in a manner that is adaptive to variations in the position and orientation of the user's face, and, moreover, that accounts for variations in illumination quality.